---
sidebar_position: 1
---

<div></div>

import openAILogo from '/img/openAILogo.png';

# <img src={openAILogo} className="adaptive-logo-filter" width="40" style={{float: 'left', marginRight: '10px', marginTop: '9px'}} /><span className="direct-service-title">OpenAI</span>

Properties used to connect to [OpenAI](https://openai.com/blog/openai-api).

### `openAI` {#openAI}

- Type: \{<br />
  &nbsp;&nbsp;&nbsp;&nbsp; [`chat?: Chat`](#Chat), <br />
  &nbsp;&nbsp;&nbsp;&nbsp; [`assistant?: Assistant`](#Assistant), <br />
  &nbsp;&nbsp;&nbsp;&nbsp; [`images?: Images`](#Images), <br />
  &nbsp;&nbsp;&nbsp;&nbsp; [`textToSpeech?: TextToSpeech`](#TextToSpeech), <br />
  &nbsp;&nbsp;&nbsp;&nbsp; [`speechToText?: SpeechToText`](#SpeechToText) <br />
  \}
- Default: _\{chat: true\}_

import ContainersKeyToggleFunction from '@site/src/components/table/containersKeyToggleFunction';
import ContainersKeyToggle from '@site/src/components/table/containersKeyToggle';
import ComponentContainer from '@site/src/components/table/componentContainer';
import DeepChatBrowser from '@site/src/components/table/deepChatBrowser';
import LineBreak from '@site/src/components/markdown/lineBreak';
import BrowserOnly from '@docusaurus/BrowserOnly';
import TabItem from '@theme/TabItem';
import Tabs from '@theme/Tabs';

<BrowserOnly>{() => require('@site/src/components/nav/autoNavToggle').readdAutoNavShadowToggle()}</BrowserOnly>

## Service Types

### `Chat` {#Chat}

- Type: `true` | \{<br />
  &nbsp;&nbsp;&nbsp;&nbsp; `system_prompt?: string`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `model?: string`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `max_tokens?: number`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `temperature?: number`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `top_p?: number`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; [`ChatFunctions`](#chat-functions) <br />
  \}
- Default: _\{system_prompt: "You are a helpful assistant.", model: "gpt-4o"\}_

Connect to Open AI's [`chat`](https://platform.openai.com/docs/api-reference/chat) API. You can set this property to _true_ or configure it using an object: <br />
`system_prompt` is used to set the [_"system"_](https://platform.openai.com/docs/api-reference/chat/create) message for the conversation context. <br />
`model` is the name of the model to be used by the API. Check [/v1/chat/completions](https://platform.openai.com/docs/models/model-endpoint-compatibility) for more. <br />
`max_tokens` the maximum number of tokens to generate in the chat. Check [tokenizer](https://platform.openai.com/tokenizer) for more info. <br />
`temperature` is used for sampling; between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused. <br />
`top_p` is an alternative to sampling with temperature, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens
comprising the top 10% probability mass are considered. <br />
[`ChatFunctions`](#chat-functions) encompasses properties used for [function calling](https://platform.openai.com/docs/guides/function-calling). <br />

#### Basic Example

<ContainersKeyToggle>
  <ComponentContainer>
    <DeepChatBrowser
      style={{borderRadius: '8px'}}
      directConnection={{
        openAI: {
          key: 'placeholder key',
          chat: {max_tokens: 2000, system_prompt: 'Assist me with anything you can'},
        },
      }}
    ></DeepChatBrowser>
  </ComponentContainer>
  <ComponentContainer>
    <DeepChatBrowser
      style={{borderRadius: '8px'}}
      directConnection={{
        openAI: {
          chat: {max_tokens: 2000, system_prompt: 'Assist me with anything you can'},
        },
      }}
    ></DeepChatBrowser>
  </ComponentContainer>
</ContainersKeyToggle>

<Tabs>
<TabItem value="js" label="Sample code">

```html
<deep-chat
  directConnection='{
    "openAI": {
      "key": "placeholder key",
      "chat": {"max_tokens": 2000, "system_prompt": "Assist me with anything you can"}
    }
  }'
></deep-chat>
```

</TabItem>
<TabItem value="py" label="Full code">

```html
<!-- This example is for Vanilla JS and should be tailored to your framework (see Examples) -->

<deep-chat
  directConnection='{
    "openAI": {
      "key": "placeholder key",
      "chat": {"max_tokens": 2000, "system_prompt": "Assist me with anything you can"}
    }
  }'
  style="border-radius: 8px"
></deep-chat>
```

</TabItem>
</Tabs>

<LineBreak></LineBreak>

#### Vision Example

If `max_tokens` is not set, the component sets it to _300_ as otherwise the API does not send a full response.

<ContainersKeyToggle>
  <ComponentContainer>
    <DeepChatBrowser
      style={{borderRadius: '8px'}}
      directConnection={{
        openAI: {
          key: 'placeholder key',
          chat: {model: 'gpt-4-vision-preview'},
        },
      }}
      images={true}
      camera={true}
      textInput={{styles: {container: {width: '77%'}}}}
    ></DeepChatBrowser>
  </ComponentContainer>
  <ComponentContainer>
    <DeepChatBrowser
      style={{borderRadius: '8px'}}
      directConnection={{
        openAI: {
          chat: {model: 'gpt-4-vision-preview'},
        },
      }}
      images={true}
      camera={true}
      textInput={{styles: {container: {width: '77%'}}}}
    ></DeepChatBrowser>
  </ComponentContainer>
</ContainersKeyToggle>

<Tabs>
<TabItem value="js" label="Sample code">

```html
<deep-chat
  directConnection='{
    "openAI": {
      "key": "placeholder key",
      "chat": {"model": "gpt-4-vision-preview"}
  }}'
  images="true"
  camera="true"
></deep-chat>
```

</TabItem>
<TabItem value="py" label="Full code">

```html
<!-- This example is for Vanilla JS and should be tailored to your framework (see Examples) -->

<deep-chat
  directConnection='{
    "openAI": {
      "key": "placeholder key",
      "chat": {"model": "gpt-4-vision-preview"}
  }}'
  images="true"
  camera="true"
  style="border-radius: 8px"
  textInput='{"styles": {"container": {"width": "77%"}}}'
></deep-chat>
```

</TabItem>
</Tabs>

<LineBreak></LineBreak>

### `Assistant` {#Assistant}

- Type: `true` | \{<br />
  &nbsp;&nbsp;&nbsp;&nbsp; `assistant_id?: string`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `thread_id?: string`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `load_thread_history?: boolean`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; [`new_assistant?: NewAssistant`](#NewAssistant), <br />
  &nbsp;&nbsp;&nbsp;&nbsp; [`files_tool_type?: FileToolTypes`](#FileToolTypes), <br />
  &nbsp;&nbsp;&nbsp;&nbsp; [`function_handler?: AssistantFunctionHandler`](#assistant-functions) <br />
  \}

Connect to your Open AI [`assistant`](https://platform.openai.com/docs/assistants/overview).
When set to `true` or the `assistant_id` is not defined, Deep Chat will automatically create a new assistant when the user sends the first message. <br />
`assistant_id` is the id of your assistant. <br />
`thread_id` allows you to communicate in the context of an already existing conversation/[thread](https://platform.openai.com/docs/api-reference/threads). <br />
`load_thread_history` toggles a preload of the previous conversation/[thread](https://platform.openai.com/docs/api-reference/threads) messages on chat initialisation. <br />
`new_assistant` defines the details for the newly created assistant. <br />
`files_tool_type` defines the type of a tool to be used to process an uploaded file. <br />
`function_handler` is the actual function used to handle the model's function response. Please navigate to [Assistant Functions](#assistant-functions) for more info. <br />

<ContainersKeyToggle>
  <ComponentContainer>
    <DeepChatBrowser
      style={{borderRadius: '8px'}}
      directConnection={{
        openAI: {
          key: 'placeholder key',
          assistant: {},
        },
      }}
    ></DeepChatBrowser>
  </ComponentContainer>
  <ComponentContainer>
    <DeepChatBrowser
      style={{borderRadius: '8px'}}
      directConnection={{
        openAI: {
          assistant: {},
        },
      }}
    ></DeepChatBrowser>
  </ComponentContainer>
</ContainersKeyToggle>

<Tabs>
<TabItem value="js" label="Sample code">

```html
<deep-chat
  directConnection='{
    "openAI": {
      "key": "placeholder key",
      "assistant": true
  }}'
></deep-chat>
```

</TabItem>
<TabItem value="py" label="Full code">

```html
<!-- This example is for Vanilla JS and should be tailored to your framework (see Examples) -->

<deep-chat
  directConnection='{
    "openAI": {
      "key": "placeholder key",
      "assistant": true
  }}'
  style="border-radius: 8px"
></deep-chat>
```

</TabItem>
</Tabs>

:::info
Returned [MessageContent](/docs/messages#MessageContent) contains a hidden property called `_sessionId` which
stores the thread id and allows conversation to continue on a new session.
:::

<LineBreak></LineBreak>

#### `NewAssistant` {#NewAssistant}

- Type: \{<br />
  &nbsp;&nbsp;&nbsp;&nbsp; `model?: string`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `name?: string`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `description?: string`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `instructions?: string`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `tools?`: \{ <br />
  &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; `type?`: `"code_interpreter"` | `"file_search"` | `"function"`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; `function?`: \{`name: string`, `description?: string`, `parameters?: object`\} <br />
  &nbsp;&nbsp;&nbsp;&nbsp; \}, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `tool_resources?`: \{ <br />
  &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; `code_interpreter?`: \{`file_ids: string[]`\}, <br />
  &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; `file_search?`: \{<br />
  &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; `vector_store_ids: string[]`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; `vector_stores?`: \{`file_ids: string[]`\} <br />
  &nbsp;&nbsp;&nbsp;&nbsp;\}\}\} <br />

- Default: _\{model: "gpt-4"\}_

When `assistant_id` is not used, this object is used to define the details of the new assistant that will be created by Deep Chat when
the user sends a new message. This object follows the [Open AI Create Asssistant API](https://platform.openai.com/docs/api-reference/assistants/createAssistant). <br />
`model` is the name of the model to be used by the API. Check the [model overview](https://platform.openai.com/docs/models/overview) for more. <br />
`name` and `description` are used to describe the new assistant. <br />
`instructions` direct the assistant's behaviour. <br />
`tool_resources` defines the resources that the assistant has access to. <br />
`tools` is an array of objects that describe the tools the assistant will have access to.
When using the `"function"` tool, you will need to also define the `function` object. <br />

<ContainersKeyToggle>
  <ComponentContainer>
    <DeepChatBrowser
      style={{borderRadius: '8px'}}
      directConnection={{
        openAI: {
          key: 'placeholder key',
          assistant: {
            new_assistant: {
              name: 'test model',
              tools: [{type: 'code_interpreter'}],
            },
          },
        },
      }}
      mixedFiles={true}
    ></DeepChatBrowser>
  </ComponentContainer>
  <ComponentContainer>
    <DeepChatBrowser
      style={{borderRadius: '8px'}}
      directConnection={{
        openAI: {
          assistant: {
            new_assistant: {
              name: 'test model',
              tools: [{type: 'code_interpreter'}],
            },
          },
        },
      }}
      mixedFiles={true}
    ></DeepChatBrowser>
  </ComponentContainer>
</ContainersKeyToggle>

<Tabs>
<TabItem value="js" label="Sample code">

```html
<deep-chat
  directConnection='{
    "openAI": {
      "key": "placeholder key",
      "assistant": {
        "new_assistant": {
          "name": "Demo Assistant",
          "tools": [{"type": "code_interpreter"}]
      }}
  }}'
></deep-chat>
```

</TabItem>
<TabItem value="py" label="Full code">

```html
<!-- This example is for Vanilla JS and should be tailored to your framework (see Examples) -->

<deep-chat
  directConnection='{
    "openAI": {
      "key": "placeholder key",
      "assistant": {
        "new_assistant": {
          "name": "Demo Assistant",
          "tools": [{"type": "code_interpreter"}]
      }}
  }}'
  mixedFiles="true"
  style="border-radius: 8px"
></deep-chat>
```

</TabItem>
</Tabs>

:::info
You can access the created `assistant_id` via `chatElementRef._activeService.rawBody.assistant_id`.
:::

<LineBreak></LineBreak>

#### `FileToolTypes` {#FileToolTypes}

- Type: [FileToolType](#FileToolType) | (`fileNames: string[]`) => [FileToolType](#FileToolType)
- Default: _"images"_

This is used to define the type of tool that will be used to process uploaded files. You can either define it as a string or a function that will return
the tool type based on the uploaded files. <br />
When nothing is defined and the user uploads an image, Deep Chat will automatically use [`"images"`](https://platform.openai.com/docs/assistants/how-it-works/creating-image-input-content)
which will not use any tools and send the image directly to the [_vision model_](https://platform.openai.com/docs/models). <br />
It is important to note that the `"code_interpreter"` and `"file_search"` tools must be toggled ON in the assistant that you are using before
the files are uploaded. This can either be done in the [OpenAI Assistant Playground](https://platform.openai.com/playground) or in the [NewAssistant](#NewAssistant) object.

<ContainersKeyToggle>
  <ComponentContainer>
    <DeepChatBrowser
      style={{borderRadius: '8px'}}
      directConnection={{
        openAI: {
          key: 'placeholder key',
          assistant: {
            assistant_id: 'assistant with code interpreter',
            files_tool_type: 'code_interpreter',
          },
        },
      }}
      mixedFiles={true}
    ></DeepChatBrowser>
  </ComponentContainer>
  <ComponentContainer>
    <DeepChatBrowser
      style={{borderRadius: '8px'}}
      directConnection={{
        openAI: {
          assistant: {
            assistant_id: 'assistant with code interpreter',
            files_tool_type: 'code_interpreter',
          },
        },
      }}
      mixedFiles={true}
    ></DeepChatBrowser>
  </ComponentContainer>
</ContainersKeyToggle>

<Tabs>
<TabItem value="js" label="Sample code">

```html
<deep-chat
  directConnection='{
    "openAI": {
      "key": "placeholder key",
      "assistant": {
        "assistant_id": "assistant with code interpreter",
        "files_tool_type": "code_interpreter"
  }}}'
></deep-chat>
```

</TabItem>
<TabItem value="py" label="Full code">

```html
<!-- This example is for Vanilla JS and should be tailored to your framework (see Examples) -->

<deep-chat
  directConnection='{
    "openAI": {
      "key": "placeholder key",
      "assistant": {
        "assistant_id": "assistant with code interpreter",
        "files_tool_type": "code_interpreter"
  }}}'
  mixedFiles="true"
  style="border-radius: 8px"
></deep-chat>
```

</TabItem>
</Tabs>

<LineBreak></LineBreak>

:::info
When uploading a file, the user must also submit a text message.
:::

<LineBreak></LineBreak>

#### `FileToolType` {#FileToolType}

- Type: `"code_interpreter"` | `"file_search"` | `"images"`

Type of tool used to process an uploaded file. Find out more information in [`"code_interpreter"`](https://platform.openai.com/docs/assistants/tools/code-interpreter) and
[`"file_search"`](https://platform.openai.com/docs/assistants/tools/file-search). [`"images"`](https://platform.openai.com/docs/assistants/how-it-works/managing-threads-and-messages) is
technically not a tool but a way to indicate that image files will be sent directly to a [_vision model_](https://platform.openai.com/docs/models).

<LineBreak></LineBreak>

### `Images` {#Images}

- Type: `true` | [`Dall-e-2`](#dall-e-2) | [`Dall-e-3`](#dall-e-3)
- Default: _Dall-e-2_

Connect to Open AI's [`Images`](https://platform.openai.com/docs/api-reference/images) API.
Set this property to _true_ or use either of the [`Dall-e-2`](#dall-e-2) or [`Dall-e-3`](#dall-e-3) objects. <br />

You can automatically call any of the following three APIs by combining different inputs:

- [Create Image](https://platform.openai.com/docs/api-reference/images/create) - Send text.
- [Create Image Variation](https://platform.openai.com/docs/api-reference/images/create-variation) - Upload and send an image with no text.
- [Create Image Edit](https://platform.openai.com/docs/api-reference/images/create-edit) - Upload an image and add text. You can also upload a second image to be used as a _mask_.

#### Example

<ContainersKeyToggle>
  <ComponentContainer>
    <DeepChatBrowser
      style={{borderRadius: '8px'}}
      directConnection={{
        openAI: {
          key: 'placeholder key',
          images: {n: 1, size: '1024x1024', response_format: 'url'},
        },
      }}
    ></DeepChatBrowser>
  </ComponentContainer>
  <ComponentContainer>
    <DeepChatBrowser
      style={{borderRadius: '8px'}}
      directConnection={{
        openAI: {
          images: {n: 1, size: '1024x1024', response_format: 'url'},
        },
      }}
    ></DeepChatBrowser>
  </ComponentContainer>
</ContainersKeyToggle>

<Tabs>
<TabItem value="js" label="Sample code">

```html
<deep-chat
  directConnection='{
    "openAI": {
      "key": "placeholder key",
      "images": {"n": 1, "size": "1024x1024", "response_format": "url"}
    }
  }'
></deep-chat>
```

</TabItem>
<TabItem value="py" label="Full code">

```html
<!-- This example is for Vanilla JS and should be tailored to your framework (see Examples) -->

<deep-chat
  directConnection='{
    "openAI": {
      "key": "placeholder key",
      "images": {"n": 2, "size": "1024x1024", "response_format": "url"}
    }
  }'
  style="border-radius: 8px"
></deep-chat>
```

</TabItem>
</Tabs>

#### `Dall-e-2`

- Type: \{<br />
  &nbsp;&nbsp;&nbsp;&nbsp; `model?: "dall-e-2"`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `n?: number`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `size?:` `"256x256"` | `"512x512"` | `"1024x1024"`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `response_format?:` `"url"` | `"b64_json"`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `user?: number` <br />
  \}
- Default: _\{model: "dall-e-2", size: "1024x1024"\}_

`model` is the name of the specific model to be used by the API. <br />
`n` is the number of images to generate. Ranges between 1 and 10. <br />
`size` is the pixel dimensions of the generated images. <br />
`response_format` is the format in which the generated images are returned. <br />
`user` is a unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. More info can be found [`here`](https://platform.openai.com/docs/guides/safety-best-practices/end-user-ids). <br />

#### `Dall-e-3`

- Type: \{<br />
  &nbsp;&nbsp;&nbsp;&nbsp; `model: "dall-e-3"`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `size?:` `"1024x1024"` | `"1792x1024"` | `"1024x1792"`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `response_format?:` `"url"` | `"b64_json"`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `user?: number` <br />
  \}
- Default: _\{size: "1024x1024"\}_

`model` is the name of the specific model to be used by the API. <br />
`size` is the pixel dimensions of the generated images. <br />
`response_format` is the format in which the generated images are returned. <br />
`user` is a unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. More info can be found [`here`](https://platform.openai.com/docs/guides/safety-best-practices/end-user-ids). <br />

<LineBreak></LineBreak>

### `TextToSpeech` {#TextToSpeech}

- Type: `true` | \{ <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `model?: string`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `voice?: string`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `speed?: number` <br />
  \}
- Default: _\{model: "tts-1", voice: "alloy", speed: 1\}_

Connect to Open AI's [`Text To Speech`](https://platform.openai.com/docs/guides/text-to-speech) API.
You can set this property to _true_ or [configure](https://platform.openai.com/docs/api-reference/audio/createSpeech) it using an object: <br />
`model` defines the target model used by the API. Check [/v1/audio/speech](https://platform.openai.com/docs/models/model-endpoint-compatibility) for more. <br />
`voice` is the name of the voice used in the generated audio. <br />
`speed` defines speed of the generated audio. It accepts a value between 0.25 and 4.0.

#### Example

<ContainersKeyToggle>
  <ComponentContainer>
    <DeepChatBrowser
      style={{borderRadius: '8px'}}
      directConnection={{
        openAI: {
          key: 'placeholder key',
          textToSpeech: {voice: 'echo'},
        },
      }}
    ></DeepChatBrowser>
  </ComponentContainer>
  <ComponentContainer>
    <DeepChatBrowser
      style={{borderRadius: '8px'}}
      directConnection={{
        openAI: {
          textToSpeech: {voice: 'echo'},
        },
      }}
    ></DeepChatBrowser>
  </ComponentContainer>
</ContainersKeyToggle>

<Tabs>
<TabItem value="js" label="Sample code">

```html
<deep-chat
  directConnection='{
    "openAI": {
      "key": "placeholder key",
      "textToSpeech": {"voice": "echo"}
    }
  }'
></deep-chat>
```

</TabItem>
<TabItem value="py" label="Full code">

```html
<!-- This example is for Vanilla JS and should be tailored to your framework (see Examples) -->

<deep-chat
  directConnection='{
    "openAI": {
      "key": "placeholder key",
      "textToSpeech": {"voice": "echo"}
    }
  }'
  style="border-radius: 8px"
></deep-chat>
```

</TabItem>
</Tabs>

<LineBreak></LineBreak>

### `SpeechToText` {#SpeechToText}

- Type: `true` | \{ <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `model?: "whisper-1"`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `temperature?: number`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `language?: string`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `type?:` `"transcription" | "translation"` <br />
  \}
- Default: _\{model: "whisper-1", type: "transcription"\}_

Connect to Open AI's [`Speech To Text`](https://platform.openai.com/docs/guides/speech-to-text) API.
You can set this property to _true_ or [configure](https://platform.openai.com/docs/api-reference/audio/createTranscription) it using an object: <br />
`model` is the name of the model to use. _"whisper-1"_ is currently the only one available. <br />
`temperature` is used for sampling; between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused. <br />
`language` is the language used the input audio. Supplying the input language in [_ISO-639-1_](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) format will improve accuracy and latency. (Only used for [_transcription_](https://platform.openai.com/docs/api-reference/audio/create) based API). <br />
`type` is used to toggle between the [_transcription_](https://platform.openai.com/docs/api-reference/audio/create) and the [_translation_](https://platform.openai.com/docs/api-reference/audio/create) APIs.
Note that [_translation_](https://platform.openai.com/docs/api-reference/audio/create) can only attempt to translate audio into _English_.

#### Example

<ContainersKeyToggle>
  <ComponentContainer>
    <DeepChatBrowser
      style={{borderRadius: '8px'}}
      directConnection={{
        openAI: {
          key: 'placeholder key',
          speechToText: {model: 'whisper-1', temperature: 0.3, language: 'en', type: 'transcription'},
        },
      }}
    ></DeepChatBrowser>
  </ComponentContainer>
  <ComponentContainer>
    <DeepChatBrowser
      style={{borderRadius: '8px'}}
      directConnection={{
        openAI: {
          speechToText: {model: 'whisper-1', temperature: 0.3, language: 'en', type: 'transcription'},
        },
      }}
    ></DeepChatBrowser>
  </ComponentContainer>
</ContainersKeyToggle>

<Tabs>
<TabItem value="js" label="Sample code">

```html
<deep-chat
  directConnection='{
    "openAI": {
      "key": "placeholder key",
      "speechToText": {"model": "whisper-1", "temperature": 0.3, "language": "en", "type": "transcription"}
    }
  }'
></deep-chat>
```

</TabItem>
<TabItem value="py" label="Full code">

```html
<!-- This example is for Vanilla JS and should be tailored to your framework (see Examples) -->

<deep-chat
  directConnection='{
    "openAI": {
      "key": "placeholder key",
      "audio": {"model": "whisper-1", "temperature": 0.3, "language": "en", "type": "transcription"}
    }
  }'
  style="border-radius: 8px"
></deep-chat>
```

</TabItem>
</Tabs>

<LineBreak></LineBreak>

## Functions {#Functions}

Examples for OpenAI's [Function Calling](https://platform.openai.com/docs/guides/function-calling) features:

### `Chat Functions` {#chat-functions}

- Type: \{<br />
  &nbsp;&nbsp;&nbsp;&nbsp; [`tools: Tools`](#Tools), <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `tool_choice?:` `"auto"` | `{type: "function", function: {name: string}}`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; [`function_handler: FunctionHandler`](#FunctionHandler) <br />
  \}

Configure the chat to call your functions via the [OpenAI Function calling API](https://platform.openai.com/docs/guides/function-calling). <br />
This is particularly useful if you want the model to analyze user's requests, check whether a function should be called, extract the relevant information
from their text and return it all in a standardized response for you to act on. <br />
`tools` defines the functions that the model can signal to call based on the user's text. <br />
`tool_choice` controls which (if any) function should be called. <br />
`function_handler` is the actual function that is called with the model's instructions.

<ContainersKeyToggleFunction></ContainersKeyToggleFunction>

<Tabs>
<TabItem value="js" label="Sample code">

```js
// using JavaScript for a simplified example

chatElementRef.directConnection = {
  openAI: {
    chat: {
      tools: [
        {
          type: 'function',
          function: {
            name: 'get_current_weather',
            description: 'Get the current weather in a given location',
            parameters: {
              type: 'object',
              properties: {
                location: {
                  type: 'string',
                  description: 'The city and state, e.g. San Francisco, CA',
                },
                unit: {type: 'string', enum: ['celsius', 'fahrenheit']},
              },
              required: ['location'],
            },
          },
        },
      ],
      function_handler: (functionsDetails) => {
        return functionsDetails.map((functionDetails) => {
          return {
            response: getCurrentWeather(functionDetails.arguments),
          };
        });
      },
    },
    key: 'placeholder-key',
  },
};
```

</TabItem>
<TabItem value="py" label="Full code">

```js
// using JavaScript for a simplified example

chatElementRef.directConnection = {
    openAI: {
      chat: {
        tools: [{
            type: "function",
            function: {
              name: "get_current_weather",
              description: "Get the current weather in a given location",
              parameters: {
                type: "object",
                properties: {
                  location: {
                    type: "string",
                    description: "The city and state, e.g. San Francisco, CA",
                  },
                  unit: {type: "string", enum: ["celsius", "fahrenheit"]},
                },
                required: ["location"],
          }}}],
        function_handler: (functionsDetails) => {
          return functionsDetails.map((functionDetails) => {
            return {
              response: getCurrentWeather(functionDetails.arguments),
        };});}
      },
      key: "placeholder-key",
    },
  };
}

function getCurrentWeather(location) {
  location = location.toLowerCase();
  if (location.includes('tokyo')) {
    return JSON.stringify({location, temperature: '10', unit: 'celsius'});
  } else if (location.includes('san francisco')) {
    return JSON.stringify({location, temperature: '72', unit: 'fahrenheit'});
  } else {
    return JSON.stringify({location, temperature: '22', unit: 'celsius'});
  }
}
```

</TabItem>
</Tabs>

<LineBreak></LineBreak>

#### `Tools` {#Tools}

- Type: \{<br />
  &nbsp;&nbsp;&nbsp;&nbsp; `type: "function" | "object"`, <br />
  &nbsp;&nbsp;&nbsp;&nbsp; `function:` \{`name: string`, `description?: string`, [`parameters: JSONSchema`](https://json-schema.org/learn/miscellaneous-examples)\} <br />
  \}[]

An array describing tools that the model may call. <br />
`name` is the name of a function. <br />
`description` is used by the model to understand what the function does and when it should be called. <br />
`parameters` are arguments that the function accepts defined in a [JSON Schema](https://json-schema.org/learn/miscellaneous-examples) (see example above). <br />
Checkout the following [guide](https://platform.openai.com/docs/guides/function-calling) for more about function calling.

:::tip
If your function accepts arguments - the `type` property should be set to _"function"_, otherwise use the following object `{"type": "object", "properties": {}}`.
:::

<LineBreak></LineBreak>

#### `FunctionHandler` {#FunctionHandler}

- Type: ([`functionsDetails: FunctionsDetails`](#FunctionsDetails)) => `{response: string}[]` | `{text: string}`

The actual function that the component will call if the model wants a response from [tools](#Tools) functions. <br />
[`functionsDetails`](#FunctionsDetails) contains information about what [tool](#Tools) functions should be called. <br />
This function should either return an array of JSONs containing a `response` property for each [tool](#Tools) function (in the same order as in `functionsDetails`)
which will feed it back into the model to finalise a response, or return a JSON containing `text` which will immediately display it in the chat
and not send any details to the model.

<LineBreak></LineBreak>

### `Assistant Functions` {#assistant-functions}

- Type: ([`functionsDetails: FunctionsDetails`](#FunctionsDetails)) => `string[]`

The [`function_handler`](#Assistant) property can be assigned with the actual function that the component will call if the model wants a response from your preconfigured assistant's functions
inside the [OpneAI assistants platform](https://platform.openai.com/assistants). <br />
[`functionsDetails`](#FunctionsDetails) contains information about what functions should be called. <br />
This function should return an array of strings defining the response for each function described in `functionDetails` (in the same order)
which will feed it back into the assistant to finalise a response.

Try it out live in the Deep Chat [Playground](/playground).

<Tabs>
<TabItem value="js" label="Sample code">

```js
// using JavaScript for a simplified example

chatElementRef.directConnection = {
  openAI: {
    assistant: {
      assistant_id: 'placeholder-id',
      function_handler: (functionsDetails) => {
        return functionsDetails.map((functionDetails) => this.getCurrentWeather(functionDetails.arguments));
      },
    },
    key: 'placeholder-key',
  },
};
```

</TabItem>
<TabItem value="py" label="Full code">

```js
// using JavaScript for a simplified example

chatElementRef.directConnection = {
  openAI: {
    assistant: {
      assistant_id: 'placeholder-id',
      function_handler: (functionsDetails) => {
        return functionsDetails.map((functionDetails) => this.getCurrentWeather(functionDetails.arguments));
      },
    },
    key: 'placeholder-key',
  },
};

function getCurrentWeather(location) {
  location = location.toLowerCase();
  if (location.includes('tokyo')) {
    return 'Good';
  } else if (location.includes('san francisco')) {
    return 'Mild';
  } else {
    return 'Very Hot';
  }
}
```

</TabItem>
</Tabs>

<LineBreak></LineBreak>

## Shared Types

Types used in [`Functions`](#Functions) properties:

### `FunctionsDetails` {#FunctionsDetails}

- Type: \{`name: string`, `arguments: string`\}[]

Array of objects containing information about the functions that the model wants to call. <br />
`name` is the name of the target function. <br />
`arguments` is a stringified JSON containing properties based on the [`parameters`](#Tools) defined for the function.
